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1. Introduction 


A Brain-Computer Interface (BCI) is a non-muscular communication channel for connecting 
the brain to a computer or another device. Currently, non-invasive BCIs transform thought- 
related changes in the electroencephalogram (EEG) online and in real time into control 
signals. In such an EEG-based BCI, specific features are extracted from brain-signals, 
transformed into a control signal, and used to restore communication to patients with 
locked-in-syndrome or to control neuroprosthesis in patients with spinal cord injuries 
(Birbaumer et al., 1999; Pfurtscheller et al., 2008b; Wolpaw et al., 2002). In addition to these 
applications, which focus on communication and control, the related field of neurofeedback 
supports feedback training in people suffering from epilepsy, autism, stroke, and emotional 
or attentional disorders (Birbaumer & Cohen, 2007). 

Today the world of BCI applications is expanding and new fields are opening. One new 
direction involves BCIs to control virtual reality (VR), including BCIs for games, or using VR 
as a powerful feedback medium to reduce the need for BCI training (Leeb et al., 2007b; Scherer 
et al., 2008). Virtual environments (VE) can provide an excellent testing ground for procedures 
that could be adapted to real world scenarios, especially for patients with disabilities. If people 
can learn to control their movements or perform specific tasks in a VE, this could justify the 
much greater expense of building physical devices such as a wheelchair or robot arm that is 
controlled by a BCI. BCIs are more and more moving out of the laboratory and becoming also 
useful for healthy users in certain situations (Nijholt et al., 2008). 

One of the first efforts to combine VR and BCI technologies was Bayliss and Ballard (2000) 
and Bayliss (2003). They introduced a VR smart home in which users could control different 
appliances using a P300-based BCI. Pineda et al. (2003) showed that immersive feedback 
based on a computer game can help people learn to control a BCI based on imagined 
movement more quickly than mundane feedback, a finding we later validated with other 
immersive feedback (Leeb et al., 2006; 2007b). Lalor et al. (2005) used a steady-state visual 
evoked potential (GSVEP)-based BCI to control a character in an immersive 3-D gaming 
environment. Recently, Leeb et al. (2007b) have reported on exploring a smart virtual 
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apartment using a motor imagery-based BCI, and Holzner et al. (2009) in an experiment for 
P300-based BCI for smart home control. 

This short overview about BCI applications in VR shows that there are many different types 
of BCIs. The next section provides a short introduction to BCIs. 


2. Definition and basic principles of a BCI 


Wolpaw et al. (2002) defined a BCI as a “...new non-muscular channel for sending messages 

and commands to the external world”. Here, we clarify this definition to emphasize that any 

BCI must have the 4 following components (Pfurtscheller et al., 2010a): 

1. Direct recording: The BCI must rely at least partly on direct measures of brain activity, 
such as through electrical potentials, magnetic fields, or hemodynamic changes. 

2. Real time processing: The signal processing must occur online and yield a 
communication or control signal. 

3. Feedback: Goal-directed feedback, about the success or failure of the control, must be 
provided to the user. 

4. Intentional control: The user must perform an intentional, goal-directed mental action 
to control, such as imagining movement or focusing on a stimulus. 

These definitions implicate that each BCI is a closed-loop system with two adaptive 

controllers: the user’s brain, which produces the signals and provides the input to the BCI; 

and the BCI itself, which analyses the brain signals and transforms them to a control signal 

as the BCI output (Figure 1). 

The EEG is the most widely used brain signal in BCIs (Mason et al., 2007), and is also the 

most common signal when using a BCI system for VR control. Two types of changes can be 

extracted from the ongoing EEG signals: one change is time and phase-locked (evoked) to an 

externally or internally-paced event, the other is non-phase-locked (induced). Evoked 

signals include event-related potentials (ERPs), including the P300 and SSVEPs (Allison et 

al., 2008). Induced signals include the dynamic amplitude changes of oscillations in different 

frequency bands (Pfurtscheller & Lopes da Silva, 1999; Pfurtscheller & Neuper, 2010). Thus, 

we can differentiate between 2 types of BCI systems. One is based on predefined mental 


motor 
imagery 


(a) 


Fig. 1. Principles of BCI systems without (a) and with external stimulation (b). Motor 
imagery is the most common mental strategy in BCIs which do not rely on external 
stimulation to generate the necessary brain activity. BCIs that do rely on external 
stimulation to elicit brain activity typically involve spatial visual attention. 
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tasks and classifying induced brain activities (Figure 1a), while the other is based on sensory 
stimulation (e.g. visual) and classifying evoked potential changes (Figure 1b). 


Three different BCI approaches have received the most attention in recent years: 


P300-based BCI (P300 BCI): The P300 is the positive component of the evoked potential that 
may develop about 300 ms after an item is flashed. The user focuses on one flashing item while 
ignoring other stimuli. Whenever the target stimulus flashes, it yields a larger P300 than the 
other possible choices. P300 BCIs are typically used to spell (Allison & Pineda, 2006; Donchin 
et al., 2000; Farwell & Donchin, 1988), but have been validated with other tasks such as control 
of a mobile robot (Bell et al., 2008) or a smart home control (Holzner et al., 2009). 

SSVEP-based BCI (SSVEP BCI): Steady-state evoked potentials (SSEPs) occur when 
sensory stimuli are repetitively delivered rapidly enough that the relevant neuronal 
structures do not return to their resting states. In a BCI application, the user focuses on one 
of several stimuli, each of which flicker at a different rate and/or phase. Flickering light 
sources are typically used to trigger steady-state visual evoked potentials. Gao et al. (2003) 
described a BCI with 48 flickering lights and a high information transfer rate of 68 bits/ min. 
This was the fastest BCI reported in the published literature until recently, when the same 
group described some improved approaches (Bin et al., 2009). Like P300 BCIs, SSVEP BCIs 
require no training and can facilitate rapid communication (Allison et al., 2008; Krusienski et 
al., 2006). SSVEP BCIs have also recently expanded to tasks beyond spelling, such as 
controlling a virtual character (avatar) in a computer game (Faller et al., 2010; Lalor et al., 
2005; Martinez et al., 2007) or controlling an orthosis (Pfurtscheller et al., 2010b). 

ERD-based BCI (ERD BCI, SMR BCI): Brain rhythms can either display an event-related 
amplitude decrease or desynchronization (ERD) or an event-related amplitude increase or 
synchronization (ERS) (Pfurtscheller & Lopes da Silva, 1999). The term ERD BCI describes any 
BCI system that relies on the detection of amplitude changes in sensorimotor rhythms (mu and 
central beta rhythms) and/or other brain oscillations, also including short-lasting post- 
imagery beta bursts (beta ERS, beta rebound) (Blankertz et al., 2007; Pfurtscheller et al., 2006a; 
2008b; Pfurtscheller & Solis- Escalante, 2009). The term SMR BCI is frequently used when only 
sensorimotor rhythms are classified (Birbaumer & Cohen, 2007). In the standard protocol, the 
user performs a motor imagery task to consciously modify brain rhythms. Motor imagery 
results in a somatotopically organised activation pattern, similar to that observed when the 
same movement is really executed (Lotze et al., 1999; Pfurtscheller & Neuper, 2010). In 
particular, hand and feet motor imagery affect sensorimotor EEG rhythms in ways that allow a 
BCI to detect such changes online and generate a reliable control signal. 

The ERD BCI can operate in two different modes: the input data are either processed in a 
predefined time windows of a few seconds each following cue stimuli (synchronous or cue- 
based BCI), or continuously sample-by-sample (asynchronous or self-paced BCI). In a 
synchronous protocol, the user performs a mental task after each cue; the EEG processing is 
time-locked to externally-paced cues that are repeated in intervals of several seconds 
because the onset of motor imagery is precisely known. No cue is presented in the 
asynchronous mode. Hence, the system is continuously available for control, allowing users 
to freely decide when they wish to generate a control signal. The output (control) signal of a 
BCI can be either the result of the user’s intended control (IC) or intended non-control 
(INC). In the latter case, during the resting state, undefined mental tasks (thoughts) or 
artifacts may be erroneously classified as control signals. Such an asynchronous BCI is more 
complex and demanding for developers, even though it may be easier for the user. 
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However, asynchronous BCIs have been validated even in advanced situations, such as 
navigating in a virtual environment (Leeb et al., 2007a; 2007d; Scherer et al., 2008). 


3. Virtual Reality system 


In order to carry out the VE experiments, two different and complex systems had to be 
integrated: the BCI and the VR system. Figure 2 shows that both systems run on two 
different machines (hardware) and different platforms (software). The interaction between 
them is realized via a network connection (usually TCP/IP). 

The participant is placed in the middle of a multi-projection based stereo and head-tracked 
VR system commonly known as a “CAVE” (Cruz-Neira et al., 1993). The surround 
projection system consists of three rear-projected active stereo screens (left, right and front 
wall on which the images are projected from outside) and a front-projected screen on the 
floor (image for the floor is projected from above), as shown in the right part of Figure 2. It 
generates three-dimensional stereoscopic representations of computer animated worlds, 
depending on the current viewing position and direction of the visitor. 

The projections on the screens are continuously adapted to the movements of the visitor by 
re-computing the projected images for the respective current viewing position and direction 
(update rate of 30 - 50 times per second). The position of the subject is determined using 
four infrared cameras that keep track of a number of highly retro-reflective balls that are 
attached to the shutter glasses. This makes it possible to compute images for every screen 
that accurately fit the visitor's view on the simulated scene. 

A special feature of any multi-wall VR system is that the images on the adjacent walls are 
seamlessly joined together, so that participants do not see the physical corners but the 
continuous virtual world that is projected with active stereo (Slater et al., 2002). The basic 
idea is to let a user become immersed in a 3-D scene (Slater & Usoh, 1993). Therefore, the 
subject has to wear shutter glasses (CrystalEyes, StereoGraphics Corporation, San Rafael, 
USA) to see the two separate stereoscopic images generated for each eye of the observer. 
The creation of the 3-D virtual environment entails two consecutive steps: first, the creation 
of a 3-D model of the scene; and second, the generation of a VR-application that controls and 
animates the modeled scene. 


contro! shutter 
glasses 


EEG acquisition 


stereoscopic 


preprocessing presentation 


feature extraction generate VE 


2 position update 
LDA classification within the VE 
post processing read tracker info 
for asynchronous 

network 


feedback generation 
connection 


get BCI input 


fs = 250 Hz fs = 120 Hz (2°60Hz) 


Fig. 2. System diagram of the hardware setup. The BCI system on the left analyses the EEG 
signals, and the extracted control commands are transferred into movements with the VE 
projected in the CAVE system. On the right bottom side is the physical CAVE installation 
with indicated projection screens. Modified from Leeb (2008). 
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4. Setup of an EEG-based BCI 


The first step is always the same. The computer (classifier) has to learn how subject-specific 
EEG data (trials) look during execution of a predefined mental task. In general, each trial 
starts with a warning stimulus followed by a cue stimulus, indicating the type of mental 
task. The task depends on whether the BCI requires external stimulation to generate the 
necessary brain signals. P300 BCI and SSVEP BCIs do require such stimuli, while ERD BCIs 
do not. In the former BCI approaches, the user typically must pay attention to one out of 2 or 
more flickering lights or to flashing letters, numbers, and/or other symbols or commands. 
In the latter case the user has to perform a motor imagery task (e.g. hand or foot movement 
imagery) as indicated by the cue stimulus. During this training, about 80 trials of EEG data 
(each lasting several seconds) are recorded without any feedback to the user and used to set 
up the classifier. In a second step, the user has to perform the cue-paced mental task, and 
real-time feedback informs the user about success or failure of the online classification. P300- 
based and SSVEP-based BCIs have an apparent advantage in that both require no or only 
minimal user training. In contrast, ERD-based BCIs often need extensive user training, 
sometimes lasting many weeks (for details see Pfurtscheller et al., 2000; 2006a). 

The experimental procedure commonly used with ERD-based BCI research can be divided 

into the following steps: 

1. The (new, naive) subject must be instructed on what exactly is going to be done. It is 
crucial that the subject performs kinesthetic motor imagery (Neuper et al., 2005). 

2. Training without feedback is done to acquire subject specific data for the imagery task 
used. 

3. The most discriminative features (e. g., frequency bands) are extracted from this data. 

4. The classifier is set up based on the features from step 3. If the classification accuracy is 
above 70 %, move on the next step, otherwise continue with step 2. This involves the 
selection of the best mental strategy, the localization of the best electrode recording sites 
and the optimal frequency bands. 

5. Training with feedback; that is, online processing of EEG signals. 

6. Classifier update, if the frequency bands have been modified or the EEG patterns have 
been changed. 

7. Online applications like virtual environments can be controlled. 

In general, multi-EEG channel recordings using various mental strategies can be performed 

at the beginning of step 2. Furthermore, offline optimization can be applied to determine the 

best mental strategies, electrode positions and frequency bands. 

A BCl-system is, in general, composed of the following components: Signal acquisition, 

signal processing (which includes preprocessing, feature extraction, classification / 

detection, and post processing), application interface and an output device or application 

(left part of Figure 2; see also Wolpaw et al., 2002). For the experiments reported here, the 

Graz-BCI system was used (for details see Pfurtscheller et al., 2006c). It consisted of a 

biosignal amplifier (gBSamp, g.tec medical engineering, Graz, Austria), one data acquisition 

card (E-Series, National Instruments Corporation, Austin, USA), and a standard personal 
computer running Windows XP operating system (Microsoft Corporation, Redmond, USA) 

The recording was handled by rtsBCI (Scherer, 2004), based on MATLAB 7.0.4 (MathWorks, 

Inc., Natick, USA) in combination with Simulink 6.2, Real-Time Workshop 6.2 and the open 

source package BIOSIG (BioSig, 2010). In the case of an ERD BCI band power (BP) features 

were estimated from the ongoing EEG by digitally band-pass filtering the EEG recordings 
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Fig. 3. General workflow of an experiment with a motor imagery-based BCI. 


(Butterworth IIR filter of order 5) and squaring and averaging the samples over the past 
second. Finally, the logarithm was computed from this time series. To distinguish between 
the two motor imagery tasks, Fisher's linear discriminant analysis (LDA, Bishop, 1995) was 
applied to the BP estimates (sample-by-sample). 

In the case of a SSVEP BCI, after calculation of the power spectrum, the first, second and 
third harmonic components are individually defined for each target frequency. For selecting 
one class, the sum of all its harmonic frequency components need to be larger than that of 
the other classes (for details see Miiller-Putz et al., 2005). 

The philosophy of the Graz-BCI is to use as few electrodes as possible for online 
experiments, which makes the BCI both, more comfortable and easier to apply, especially 
for subjects out of laboratory (e.g. at home or in working space). In all of the 4 experiments 
reported, EEG was recorded with an electrode cap (Easycap, Germany) fitted to the subject's 
head. EEG electrodes were mounted bipolarly over the sensorimotor cortex (ERD BCI) or 
over the occipital cortex (GSVEP BCI). The EEG was amplified (power-line notch filter was 
activated), band pass filtered between 0.5 and 100 Hz (EEG acquisition and preprocessing 
block, see Figure 2, left part) and recorded with a sampling frequency of 250 Hz. 


5. Walking from thought - impact of immersive environments 


The goals of the first study were: (i) to demonstrate that it is possible to move forward - “to 
walk” - within a VE (e.g. a virtual street) without any muscular activity, using only the 
imagination of movements recorded with a BCI and (ii) to compare the influences of 
different feedback types (common BCI feedback versus VE feedback (HMD and CAVE)) on 
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the BCI performance. Therefore, the results from three different feedback (FB) conditions 
were compared: First, the results of the standard BCI with a simple bar graph; second, using 
a head-mounted display (HMD) as a FB device; and finally, using a highly immersive 
“CAVE?” projection environment (see Figure 4a). In case of the HMD and CAVE conditions, 
the idea was to use the imagination of foot movement to walk through the VE, based on a 
synchronous BCI paradigm. The subject was instructed by a cue to imagine a right hand 
movement (single beep) or a foot movement (double beep). Three healthy volunteers 
participated several times over 5 months in this study. The task given to each participant 
was to walk to the end of a virtual street (see Figure 4a), and only in the case of successful 
foot motor imagery would a motion occur (for further details see Leeb et al., 2006; 
Pfurtscheller et al., 2006a). Correct classification of foot motor imagery was accompanied by 
forward movement at a constant speed in the virtual street, whereas a correct classification 
of hand motor imagery stopped the motion. Incorrect classification of foot motor imagery 
resulted in halting, and incorrect classification of hand motor imagery in backward motion 
(same speed). The walking distance was scored as “cumulative achieved mileage” (CAM, 
Leeb & Pfurtscheller, 2004; Leeb et al., 2006), which was the integrated forward/backward 
distance covered during foot movement imagination and was used as the performance 
measurement. So the BCI output of the online classification was either used to control the 
length and orientation of the bar graph feedback (control condition) or to move through a 
virtual street (HMD or CAVE condition). The CAM performances of the bar graph feedback 
experiments were simulated offline to enable comparison. 


Type of feedback 


= bar graph 
HMD 
== CAVE 
H C B H C 
t [trials] S1 $2 $3 


Fig. 4. (a) Participant in the virtual main street with shops and animated avatars. The subject 
wears an electrode cap and shutter glasses. (b) Examples of task performances displayed in 
the theoretical possible CAM (dashed line) and the real CAM (full line) of one run of one 
subject. The cue class indicated is written above the line. Due to the random cue sequence, 
each participant had a different theoretical pathway (dashed line). (c) Boxplot of all achieved 
CAMs of all subjects and feedback types. The diagram consists of 3 groups, each 
corresponding to a subject. Within these groups the left plots corresponds to standard bar- 
graph feedback (B), the middle to HMD feedback (H) and the right one to CAVE feedback 
(C). Modified from Leeb et al. (2007c). 
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The CAM of an example result of subject S1 (session 2, run 4) is plotted in Figure 4b. Both 
the theoretically possible CAM (dashed line) and the real achieved CAM (full line) are 
plotted. A CAM of 100 % corresponds to a correct classification of all 40 imagery tasks over 
the entire feedback time. A random classification would result in an expected CAM of 0 %. 
It is almost impossible to achieve the maximum attainable CAM of 100 %, because even a 
small procrastination or hesitation reduces mileage. 

All the subjects were able to walk through the virtual street, and the resulting BCI 
performance in the VR tasks was comparable to standard BCI recordings. The use of VR as 
feedback stimulated the participant's performances and provided motivation. All the 
subjects achieved their best results within the CAVE and the worst in the standard BCI 
conditions. The mean achieved CAM of all participants and condition is plotted in Figure 4c. 
Two participants showed an increase over the condition, but participant S1 achieved worse 
results with the HMD (for further details see Leeb et al., 2006). 

These data indicate that foot motor imagery is a suitable mental strategy to control events 
within the VEs. Imagination of feet movement is a mental task which comes very close to 
that of natural walking. In the CAVE condition (highest immersion) the performance of two 
participants was especially outstanding (up to 100 % BCI classification accuracy of single 
trials), although we observed a variability in the classification results between individual 
runs. 


6. Exploring a virtual apartment using an ERD-based BCI 


The second study shows that ten naive subjects can be trained in a synchronous paradigm 
within 3 sessions to navigate freely through a virtual apartment (see Figure 5a), whereby, at 
every junction, the subjects could decide on their own how they wanted to explore the VE. 
The very important and crucial step away from a synchronized or cue-based BCI and from 
laboratory conditions towards real world applications is performed in this experiment. The 
virtual apartment was similarly designed to a real world application, with a goal-oriented 
task, a high mental workload and a variable decision period for the subject. All the subjects 
were able to perform long and stable motor imagery over a minimum time of two seconds 
(for further details see Leeb et al., 2007b). 

In this paradigm, only the start of the decision period was indicated with a “neutral” cue 
consisting of two arrows (see Figure 5b). The subject could decide for him/herself which 
motor imagery he/she wanted to perform and therefore which direction he/she wanted to 
select, but walking was only possible along predefined pathways through the corridors or 
rooms. The subject received feedback by viewing the size of the arrows, which were 
modulated depending on the BCI classification output. In this case, the corresponding arrow 
was huge and the subject was turned to the right/left/straight. Afterwards, the system 
automatically guided the subject to the next junction. As stated above, a “neutral cue” was 
used to indicate the starting point of the decision period (similar to the cue-based BCI), but 
the neutral cues were completely embedded in the given task and the duration of the 
decision periods (similar to trials) was variable, depending only on the performance of the 
subject. The subjects were instructed to go to a predefined target room. A small flag pole on 
the map indicated the destination which should be reached by using the shortest route 
through the maze-like apartment (see Figure 5a). The performance error was calculated by 
dividing the number of wrong decisions by the total number of junctions. 
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Each of the ten participating subjects performed 12 runs of variable duration in the virtual 
apartment. All the runs started at the same point (entrance door). During the first run, no 
instructions were given to the subjects, so they could walk freely through the apartment for 
5 minutes to become familiar with the VE. In all the other runs, the subjects were instructed 
to go to a predefined target room. A small flag pole on the map indicated the destination 
which should be reached by using the shortest route through the maze-like apartment (see 
Figure 5a). Only one target was given in the first two runs, but in further runs, the number 
of targets was increased and only one target was visible each time. If this target was 
reached, either the follow-up target was inserted or the run was finished. Each run was 
limited to 5 minutes in total. 

For comparison, synchronous BCI sessions with a standard bar-graph feedback were 
performed before and after the sessions with the virtual apartment, whereby the 
experiments with the virtual apartment were performed in front of a normal TFT monitor 
(TFT) and within an immersive virtual environment (iVE). In Figure 5c, the mean and 
standard error over all subjects is given and the statistical differences between the sessions 
are marked. All the subjects were able to deal with the variable trial length (the duration of 
the trial depended on how fast or slow the subject could perform a decision) and the 
variable inter-trial interval. The subjects noted that the task in the virtual apartment was 
much harder compared to the prior feedback training, because it was not only necessary to 
perform the “correct” imagination, but also the shortest way through the apartment had to 
be found. Therefore, the cognitive load was much higher compared to the standard BCI 
paradigm (for further details see Leeb et al., 2007b). According to our hypothesis, we found 
that the performance improves (that is, reduced error) over the sessions, and the lowest 
error was found during the sessions with virtual feedback. The slight but stable 
performance improvement of all subjects is very well known as the training effect 
(Pfurtscheller & Neuper, 2001; Wolpaw et al., 2002). 
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Fig. 5. (a) View into the virtual apartment, with one possible pathway plotted in white. The 
target room is marked with a small flag pole (in this example, the room at the upper end of 
the apartment). (b) First-person view of the virtual apartment with two arrows indicating 
the possible directions to go (“neutral cue”). (c) Mean + standard error (SE) over all subjects 
of the integrated classification error and performance error. The asterisk (* p<0.05) shows 
statistically significant post-hoc differences. Cb_1 and cb_2 are the two sessions with 
standard BCI feedback before and after the VR sessions. TFT is the session with VR feedback 
presented on a normal TFT monitor and iVE in an immersive environment. Modified from 
Leeb (2008). 
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7. Exploring a virtual apartment using an SSVEP-based BCI 


Another study involving seven healthy participants (see Faller et al., 2010) presented an 
even more flexible, asynchronous navigation paradigm within the same apartment scenario 
as in Leeb et al. (2007b) using an SSVEP-based BCI. The participants were instructed to 
navigate an avatar to two waypoints (see Figure 6a) along a given path in two runs, by 
alternately focusing attention on one of three visual stimuli that were flickering at the 
different frequencies 12, 15 and 20 Hz. Successful classifications of the according classes 
triggered the associated commands (i) turn 45° left (ii) turn 45° right and (iii) walk one step 
ahead. 

In contrast to the work presented in Chapter 6, this system requires minimal setup and 
training time and offers faster, more accurate control, with the trade-off that it requires the 
user to visually fixate certain stimuli in order to communicate a control signal. The stimuli 
were presented directly within the 3D environment (see Figure 6b). Compared to systems 
that rely on external stimuli (e.g. LEDs), this approach is more dynamic and allows the user 
to better focus on goal-directed interactions and deal with a higher mental workload in the 
task. 

Six out of seven subjects were able to navigate to the first waypoint in the first run halfway 
through the apartment. Five participants successfully reached the second waypoint in the 
second run. The average positive predictive value (precision, PPV = TP/(TP+FP), whereby 
TP stands for true positive and FP for false positive detections) over all participants was 91.7 
+ 9.9 %, which resulted in an average of 6.5 + 2.3 TP and 0.4 + 0.4 FP activations per minute 
given a dwell-time of 1.5 s and a refractory period of 4s. These fixed temporal restrictions do 
of course limit the number of possible control commands per minute. However, some 
factors other than speed may be more important for the user. For instance, choosing a higher 
refractory period (Townsend et al., 2004) makes the SSVEP BCI easier to use, more likely to 
provide effective communication for a broader population of users, more reliable, and less 
fatiguing. This system demonstrates a virtual feedback environment that allows both 
disabled and healthy users to seamlessly communicate and interact through an intuitive, 
natural and friendly interface. 


Fig. 6. (a) Overview of the apartment showing the path along the two waypoints (marked by 
the arrows) that the participants are supposed to reach in the runs 1 and 2 respectively. (b) 
Screenshot of an in-game scene, showing the avatar in third person perspective along with 
the three navigation stimuli on the left, the right and over the avatar’s head. 
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8. Moving a wheelchair in VR 


Finally, we report on a 35 year old male tetraplegic subject. After a traumatic injury of the 
spinal cord in 1998, he has a complete motor and sensory lesion below C5 and an incomplete 
lesion below C4. During an intensive training period of approximately 4 months, he learned 
to control the cue-based Graz-BCI (details about this training are reported elsewhere 
(Pfurtscheller et al., 2000). Specifically, the midcentral focused beta oscillations with a 
dominant frequency of approximately 17 Hz allowed a brain-switch like application and 
control of a VE (recorded close to Cz, foot representation area). Only one single EEG channel 
was recorded bipolarly at Cz (foot representation area). One single logarithmic band power 
feature was estimated from the ongoing EEG (see Figure 7b-c). A simple threshold (TH) was 
used to distinguish between foot movement imagination (IC) and rest (INC). For further 
details see Leeb et al. (2007a). 

The tetraplegic participant was placed with his wheelchair in the middle of a multi- 
projection based stereo VR system (“CAVE”). The VE used was a virtual street with shops 
on both sides (Leeb et al., 2006) and populated with 15 avatars, who were lined up along the 
street (see Figure 7a). The participant was instructed to “move” from avatar to avatar 
towards the end of the virtual street (65 length units) by movement imagination of his 
paralyzed feet. Only when foot MI was detected (IC) did the subject move forward (see 
Figure 7d, walking speed 1.25 units/second). Every time he was about to pass an avatar, he 
had to stop very close to it. The avatar started talking to the subject if he was standing close 
and still to it for one second. Each avatar was surrounded by an invisible communication 
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Fig. 7. (a) Picture sequence before, during and after the contact with an avatar. (b) Band-pass 
filtered (15-19 Hz) EEG and (c) logarithmic band power time course with threshold. (d) 
Indicated periods of moving and stopping. The contact with the avatar occurred at second 
11.4. Modified from Leeb (2008). 
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sphere (0.5 - 2.5 units) and the subject had to stop within this sphere. The size of the sphere 
approximated the distance for a conversation in the real world. The avatar started talking to 
the subject if he was standing still for one second within this sphere. After finishing a 
randomly chosen short statement (like: “Hi”, “My name is Maggie”, “It was good to meet 
you”...), the avatar walked away. Communication was only possible within the sphere; if 
the subject stopped too early or stopped too close to the avatar, nothing happened. After a 
while, and with his own free will, the subject could imagine another foot movement and 
started moving again towards the next avatar, until the end of the street was reached. 

The tetraplegic subject performed ten runs on two days, and he was able to stop at 90 % of 
the 150 avatars and talk to them. In four runs, he achieved a performance of 100 %. In 11 of 
the 15 missed avatars (of all runs), the subject stopped within the communication range, but 
the stopping time was too short (between 0.08 to 0.88 s mean + SD = 0.47 s + 0.27 s). 

In an interview after the experiment, the patient confirmed that “moving” occurred only 
during periods of foot motor imagery, but he reported that it was hard to stop precisely. 
When the avatars were placed very laterally, it was especially hard to find the correct 
distance to the avatar. Concerning the experience with the interaction, he mentioned that “It 
has never happened before, in the sense of success and interaction. I thought that I was on the street 
and I had the chance to walk up to the people. I just imagined the movement and walked up to them. 
However, I had the sensation that they were just speaking but not talking to me...” He said that he 
had the feeling of being in that street and forgot that he was in the lab and people were 
around him. “Of course the image on the CAVE wall didn’t look like you or me, but it still felt as if I 
was moving in a real street, not realistic, but real. I checked the people (avatars). We had 14 ladies 
and 1 man.” The subject stated that he felt surprised as one avatar walked through him; he 
wanted to get out of the way, to go backwards. This suggests that the subject felt very 
absorbed in the virtual reality environment. 

This work demonstrated for the first time that a tetraplegic subject, sitting in a wheel chair, 
could control his movements in a VE by the usage of a self-paced (asynchronous) BCI based 
on one single EEG recording (for further details see Leeb et al., 2007a). The mentally induced 
beta oscillation in our patient is a unique phenomenon and probably the result of the 
intensive and long-lasting BCI feedback training with the goal of achieving control over 
brain waves. The use of visually rich and stimulating VE, involving avatars that were 
interacting with the subject, accounted for a diverse and challenging experimental 
paradigm. The simulation power of the VE ensured that he had the sense of being in the 
street and going to the people; therefore the experiment was similar to a task in a real street. 

VEs are especially attractive for a person who is wheelchair-bound. First, simply using a VE 
can give such persons access to experiences that may be long forgotten (or which they have 
never had). The fact that the subject could still perform feet motor imagery, years after an 
injury that rendered him unable to use his feet, is a testament to the plasticity of the human 
brain (see also Kiibler et al., 2005). Another advantage here is that VEs can be used to create 
virtual prototypes of new navigation or control methods, and give potential users 
experience of them in a safe environment before they are ever physically built. 


9. Importance of VR feedback in BCI research 


The increasing availability of VR technology has generated a high interest in studying BCI 
interaction with VEs. 
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Compared to traditional, abstract interfaces like mouse or keypad, BCI systems could 
potentially promise a more direct and intuitive way of interacting and thereby overcome 
some limitations of navigating within VEs (Usoh & Slater, 1995). This is especially obvious 
for stimulus-dependent BCI-VR systems (Figure 1b), where users can control appliances in the 
VE by simply directing their eye gaze and/or focus of attention towards the desired element 
(e.g., looking at the TV to switch it on, looking at the door to open it; Allison et al., 2008). In 
a typical independent BCI paradigm (Figure 1a) as described above, users learn to direct, e.g., 
a computer cursor towards a highlighted target via certain changes of specific EEG features. 
To accomplish this task, each participant develops a specific mental strategy to modify the 
relevant brain signals and use a BCI effectively. As ERD BCIs mostly rely on motor imagery, 
the mental process of imagining different types of movements offers an intuitive way of VE 
control like, for example, imagining foot movements for moving forward in a VE (Leeb et 
al., 2007a; Pfurtscheller et al., 2006a). A further advantage of using sensorimotor rhythms in 
BCI-based VE control is that they are typically modulated by motor activity (including both 
actual and imagined movements), but unaffected by changes in visual stimulation. 
Therefore people can use an ERD BCI while performing other visual tasks. Indeed, we have 
successfully shown that people can use an ERD BCI (based on motor imagery) at the same 
time as an SSVEP BCI (based on visual attention) (Allison et al., 2010; Brunner et al., 2010; 
Pfurtscheller et al., 2010a; 2010b). 

Furthermore, research that explores BCI training in VEs is important for many reasons. VEs 
can help focus the training on the specific target application. For example, different training 
settings and paradigms may be more or less efficient, depending on whether the user’s task 
is to select certain characters or icons for communication, control a neuroprosthesis to 
restore grasping, or navigate a wheelchair in a realistic environment. Moreover, it is a matter 
of ongoing research whether BCI training in stimulus-rich virtual environments is more 
effective than the standard training procedure using simple, more or less abstract, visual 
cues and feedback stimuli. The literature suggests that a realistic VE enhances the feeling of 
presence, task performance and also cortical activation (Jancke et al., 2009; Lee and Kim, 
2008; Slater et al., 2002). Combining BCI and VR technologies can lead to highly realistic and 
immersive BCI feedback scenarios that make participants more engaged and motivated. 
Thus, it is not unexpected that a rich visual representation of the feedback signal, such as a 
3-dimensional video game or VE, may facilitate learning to use a BCI (Leeb, 2008; Pineda et 
al., 2003; Ron-Angevin et al., 2005). 

The feedback is a very important component of the BCI as it provides the user with 
information about the efficiency of his/her strategy and enables learning. The studies 
mentioned above support realistic and engaging feedback scenarios which are closely 
related to the specific target application. However, the processing of such a realistic 
feedback stimulus may also interfere with the mental motor imagery task, and thus might 
impair the development of BCI control (Neuper et al., 2009). This could be the case when the 
realistic feedback presentation showing (for instance) a moving visual scene is not 
compatible with the specific mental action the user creates to induce the relevant brain 
signals. Therefore, the mutual interaction between a mentally simulated movement (for BCI 
control) and simultaneous watching of a moving scene should be further investigated in 
future studies. 

Another interesting aspect is that mental simulation of movement (motor imagery) results in 
cardiovascular changes explained by two factors, i.e. function of anticipation of movement 
and central preparation of movement (Damen & Brunia, 1987; Oishi et al., 2000). The heart 
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rate (HR) generally decreases during motor imagery in laboratory conditions without VR 
feedback (Leeb, 2008; Pfurtscheller et al., 2006b), but can be increased during effortful 
imagery and/or enhanced mental effort (Decety et al., 1991), such as during VR feedback in 
BCI experiments (Pfurtscheller et al., 2006b; 2008a). This underlines the importance of VR 
feedback in modifying emotional experiences and enhances autonomic and visceral 
responses. The HR changes can be in the order of several beats per minute and therewith 
used to increase the classification accuracy of an ERD-based BCI when both the EEG and the 
HR are analyzed simultaneously. 


10. Concluding remarks 


One important component of a BCI system is feedback. VR offers a powerful possibility not 
only to rehearse scenarios that are otherwise too dangerous or expensive to set up, such as 
simulation of wheel chair movement, but also to modify emotional experiences and enhance 
therewith autonomic responses as e.g. heart rate changes. This opens a new way to improve 
the performance of a BCI by simultaneously analyzing and classifying brain and heart rate 
changes and to realize a “hybrid BCI” (Pfurtscheller et al., 2010a). Furthermore, VR 
environments may offer numerous other benefits, such as reduced training time, improved 
classification accuracy, increased sense of immersion/presence in an artificial setting, and 
reduced boredom or fatigue. 


11. Acknowledgment 


This work was carried out as part of the “PresenCCia” project, an EU funded Integrated 
Project under the IST program (project no. 27731) and the EU funded project “BrainAble” 
(project no. ICT-2010-247447). The authors are grateful to Brendan Allison for proofreading 
the manuscript. 


12. References 


Allison, B. Z.; Brunner, C.; Kaiser, V.; Miiller-Putz, G. R.; Neuper, C. & Pfurtscheller, G. 
(2010). Toward a Hybrid Brain-Computer Interface Based on Imagined Movement 
and Visual Attention, J Neural Eng, Vol. 7, No. 2, 26007 

Allison, B. Z.; McFarland, D. J.; Schalk, G.; Zheng, S. D.; Jackson, M. M. & Wolpaw, J. R. 
(2008). Towards an Independent Brain-Computer Interface Using Steady State 
Visual Evoked Potentials, Clin Neurophysiol, Vol. 119, No. 2, 399-408 

Allison, B. Z. & Pineda, J. A. (2006). Effects of SOA and Flash Pattern Manipulations on 
ERPs, Performance, and Preference: Implications for a BCI System, Int J 
Psychophysiol, Vol. 59, No. 2, 127-40 

Bayliss, J. D. (2003). Use of the Evoked Potential P3 Component for Control in a Virtual 
Apartment, IEEE Trans Neural Syst Rehabil Eng, Vol. 11, No. 2, 113-6 

Bayliss, J. D. & Ballard, D. H. (2000). A Virtual Reality Testbed for Brain-Computer Interface 
Research, IEEE Trans Rehabil Eng, Vol. 8, No. 2, 188-90 

Bell, C. J.; Shenoy, P.; Chalodhorn, R. & Rao, R. P. (2008). Control of a Humanoid Robot by a 
Noninvasive Brain-Computer Interface in Humans, J Neural Eng, Vol. 5, No. 2, 214- 
20 


www.intechopen.com 


Brain-Computer Interface Systems used for Virtual Reality Control 17 


Bin, G.; Gao, X.; Yan, Z.; Hong, B. & Gao, S. (2009). An Online Multi-Channel SSVEP-Based 
Brain-Computer Interface Using a Canonical Correlation Analysis Method, J Neural 
Eng, Vol. 6, No. 4, 046002 

BioSig (2003-2010). An Open Source Software Package for Biomedical Signal Processing 
under Matlab, Available online at http://biosig.sf.net/ 

Birbaumer, N. & Cohen, L. G. (2007). Brain-Computer Interfaces: Communication and 
Restoration of Movement in Paralysis, J Physiol, Vol. 579, No. Pt 3, 621-36 

Birbaumer, N.; Ghanayim, N.; Hinterberger, T.; Iversen, I.; Kotchoubey, B.; Kübler, A.; 
Perelmouter, J.; Taub, E. & Flor, H. (1999). A Spelling Device for the Paralysed, 
Nature, Vol. 398, No. 6725, 297-8 

Bishop, C. M. (1995). Neural Networks for Pattern Recognition. Clarendon Press, Oxford, UK 

Blankertz, B.; Dornhege, G.; Krauledat, M.; Miiller, K. R. & Curio, G. (2007). The Non- 
Invasive Berlin Brain-Computer Interface: Fast Acquisition of Effective 
Performance in Untrained Subjects, Neuroimage, Vol. 37, No. 2, 539-50 

Brunner, C.; Allison, B. Z.; Krusienski, D. J.; Kaiser, V.; Miiller-Putz, G. R.; Pfurtscheller, G. 
& Neuper, C. (2010). Improved Signal Processing Approaches in an Offline 
Simulation of a Hybrid Brain-Computer Interface, J Neurosci Methods, Vol. 188, No. 
1, 165-73 

Cruz-Neira, C.; Sandin, D. J. & DeFanti, T. A. (1993). Surround-Screen Projection-Based 
Virtual Reality: The Design and Implementation of the Cave, Proceedings of the 20th 
annual conference on Computer graphics and interactive techniques, 135-42 

Damen, E. J. & Brunia, C. H. (1987). Changes in Heart Rate and Slow Brain Potentials 
Related to Motor Preparation and Stimulus Anticipation in a Time Estimation Task, 
Psychophysiology, Vol. 24, No. 6, 700-13 

Decety, J.; Jeannerod, M.; Germain, M. & Pastene, J. (1991). Vegetative Response During 
Imagined Movement Is Proportional to Mental Effort, Behav Brain Res, Vol. 42, No. 
1,1-5 

Donchin, E.; Spencer, K. M. & Wijesinghe, R. (2000). The Mental Prosthesis: Assessing the 
Speed of a P300-Based Brain-Computer Interface, IEEE Trans Rehabil Eng, Vol. 8, 
No. 2, 174-9 

Faller, J.; Miiller-Putz, G. R.; Schmalstieg, D. & Pfurtscheller, G. (2010). An Application 
Framework for Controlling an Avatar in a Desktop Based Virtual Environment via 
a Software SSVEP Brain-Computer Interface, PRESENCE Special Issue, "Brain- 
Computer Interface (BCI) Systems in Virtual Reality Environments." Vol. 19, No. 1, 25- 
34 

Farwell, L. A. & Donchin, E. (1988). Talking Off the Top of Your Head: Toward a Mental 
Prosthesis Utilizing Event-Related Brain Potentials, Electroencephalogr Clin 
Neurophysiol, Vol. 70, No. 6, 510-23 

Gao, X.; Xu, D.; Cheng, M. & Gao, S. (2003). A BCI -Based Environmental Controller for the 
Motion-Disabled, IEEE Trans Neural Syst Rehabil Eng, Vol. 11, No. 2, 137-40 

Holzner, C.; Guger, C.; Edlinger, G.; Gronegress, C. & Slater, M. (2009). Virtual Smart Home 
Controlled by Thoughts, Proc. of the 2009 18th IEEE International Workshops on 
Enabling Technologies: Infrastructures for Collaborative Enterprises - Volume 00, 236-39 

Jancke, L.; Cheetham, M. & Baumgartner, T. (2009). Virtual Reality and the Role of the 
Prefrontal Cortex in Adults and Children., Front Neurosci., Vol. 3, No. 1, 52-59 


www.intechopen.com 


18 Virtual Reality 


Krusienski, D. J.; Sellers, E. W.; Cabestaing, F.; Bayoudh, S.; McFarland, D. J.; Vaughan, T. M. 
& Wolpaw, J. R. (2006). A Comparison of Classification Techniques for the P300 
Speller, J Neural Eng, Vol. 3, No. 4, 299-305 

Kübler, A.; Nijboer, F.; Mellinger, J.; Vaughan, T. M.; Pawelzik, H.; Schalk, G.; McFarland, D. 
J.; Birbaumer, N. & Wolpaw, J. R. (2005). Patients with Als Can Use Sensorimotor 
Rhythms to Operate a Brain-Computer Interface, Neurology, Vol. 64, No. 10, 1775-7 

Lalor, E.; Kelly, S.; Finucane, C.; Burke, R.; Smith, R.; Reilly, R. B. & McDarby, G. (2005). 
Steady-State Vep-Based Brain-Computer Interface Control in an Immersive 3-D 
Gaming Environment, EURASIP J on Appl Sig Proc, Vol. 19, 3156-64 

Lee, S. & Kim, G. J. (2008). Effects of Visual Cues and Sustained Attention on Spatial 
Presence in Virtual Environments Based on Spatial and Object Distinction, Interact. 
Comput., Vol. 20, No. 4-5, 491-502 

Leeb, R. (2008). Brain-Computer Communication: The Motivation, Aim, and Impact of Virtual 
Feedback. Graz University of Technology, PhD thesis 

Leeb, R.; Friedman, D.; Miiller-Putz, G. R.; Scherer, R.; Slater, M. & Pfurtscheller, G. (2007a). 
Self-Paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A 
Case Study with a Tetraplegic, Computational Intelligence and Neuroscience, special 
issue: "Brain-Computer Interfaces: Towards Practical Implementations and Potential 
Applications", Vol. 2007, No. Article ID 79642, 1-8 

Leeb, R.; Keinrath, C.; Friedman, D.; Guger, C.; Scherer, R.; Neuper, C.; Garau, M.; Antley, 
A.; Steed, A.; Slater, M. & Pfurtscheller, G. (2006). Walking by Thinking: The 
Brainwaves Are Crucial, Not the Muscles! Presence Teleoper Virtual Environ., Vol. 15, 
No. 5, 500-14 

Leeb, R.; Lee, F.; Keinrath, C.; Scherer, R.; Bischof, H. & Pfurtscheller, G. (2007b). Brain- 
Computer Communication: Motivation, Aim and Impact of Exploring a Virtual 
Apartment, IEEE Trans Neural Syst Rehabil Eng, Vol. 15, No. 4, 473-82 

Leeb, R. & Pfurtscheller, G. (2004). Walking through a Virtual City by Thought, Proc. of the 
26th A Int Conf IEEE EMBS - EMBC 2004, San Francisco, USA, 4503-6 

Leeb, R.; Scherer, R.; Friedman, D.; Lee, F.; Keinrath, C.; Bischof, H.; Slater, M. & 
Pfurtscheller, G. (2007c). Combining BCI and Virtual Reality: Scouting Virtual 
Worlds, In: Toward Brain-Computer Interfacing, edited by G. Dornhege, J. del R. 
Millan, T. Hinterberger, D. J. McFarland & K.-R. Miiller, 393-408. MIT Press, 
Cambridge, MA, USA 

Leeb, R.; Settgast, V.; Fellner, D. W. & Pfurtscheller, G. (2007d). Self-Paced Exploring of the 
Austrian National Library through Thought, Int J Bioelectromagnetism, Vol. 9, No. 4, 
237-44 

Lotze, M.; Montoya, P.; Erb, M.; Hulsmann, E.; Flor, H.; Klose, U.; Birbaumer, N. & Grodd, 
W. (1999). Activation of Cortical and Cerebellar Motor Areas During Executed and 
Imagined Hand Movements: An Fmri Study, J Cogn Neurosci, Vol. 11, No. 5, 491-501 

Martinez, P.; Bakardjian, H. & Cichocki, A. (2007). Fully Online Multicommand Brain- 
Computer Interface with Visual Neurofeedback Using SSVEP Paradigm, Comput 
Intell Neurosci, 94561 

Mason, S. G.; Bashashati, A.; Fatourechi, M.; Navarro, K. F. & Birch, G. E. (2007). A 
Comprehensive Survey of Brain Interface Technology Designs, Ann Biomed Eng, 
Vol. 35, No. 2, 137-69 


www.intechopen.com 


Brain-Computer Interface Systems used for Virtual Reality Control 19 


Miiller-Putz, G. R.; Scherer, R.; Brauneis, C. & Pfurtscheller, G. (2005). Steady-State Visual 
Evoked Potential (SSVEP)-Based Communication: Impact of Harmonic Frequency 
Components, J Neural Eng, Vol. 2, No. 4, 123-30 

Neuper, C.; Scherer, R.; Reiner, M. & Pfurtscheller, G. (2005). Imagery of Motor Actions: 
Differential Effects of Kinesthetic and Visual-Motor Mode of Imagery in Single- 
Trial EEG, Cogn Brain Res, Vol. 25, No. 3, 668-77 

Neuper, C.; Scherer, R.; Wriessnegger, S. & Pfurtscheller, G. (2009). Motor Imagery and 
Action Observation: Modulation of Sensorimotor Brain Rhythms During Mental 
Control of a Brain-Computer Interface, Clin Neurophysiol, Vol. 120, No. 2, 239-47 

Nijholt, A.; Tan, D.; Pfurtscheller, G.; Brunner, C.; Millan Jdel, R.; Allison, B. Z.; Graimann, 
B.; Popescu, F.; Blankertz, B. & Miiller, K. R. (2008). Brain-Computer Interfacing for 
Intelligent Systems, JEEE Intelligent Systems, Vol. 23, No. 3, 72-79 

Oishi, K.; Kasai, T. & Maeshima, T. (2000). Autonomic Response Specificity During Motor 
Imagery, J Physiol Anthropol Appl Human Sci, Vol. 19, No. 6, 255-61 

Pfurtscheller, G.; Allison, B. Z.; Brunner, C.; Bauernfeind, G.; Solis-Escalante, T.; Scherer, R.; 
Zander, T. O.; Miiller-Putz, G.; Neuper, C. & Birbaumer, N. (2010a). The Hybrid 
BCI, Front Neurosci, Vol. 4, 42 

Pfurtscheller, G.; Guger, C.; Miiller, G.; Krausz, G. & Neuper, C. (2000). Brain Oscillations 
Control Hand Orthosis in a Tetraplegic, Neurosci Lett, Vol. 292, No. 3, 211-4 

Pfurtscheller, G.; Leeb, R.; Friedman, D. & Slater, M. (2008a). Centrally Controlled Heart 
Rate Changes During Mental Practice in Immersive Virtual Environment: A Case 
Study with a Tetraplegic, Int J Psychophysiol, Vol. 68, No. 1, 1-5 

Pfurtscheller, G.; Leeb, R.; Keinrath, C.; Friedman, D.; Neuper, C.; Guger, C. & Slater, M. 
(2006a). Walking from Thought, Brain Res, Vol. 1071, No. 1, 145-52 

Pfurtscheller, G.; Leeb, R. & Slater, M. (2006b). Cardiac Responses Induced During Thought- 
Based Control of Virtual Environments, Int J Psychophysiol, Vol. 62, No. 1, 134-40 

Pfurtscheller, G. & Lopes da Silva, F. H. (1999). Event-Related EEG /MEG Synchronization 
and Desynchronization: Basic Principles, Clin Neurophysiol, Vol. 110, No. 11, 1842-57 

Pfurtscheller, G.; Miiller-Putz, G. R.; Scherer, R. & Neuper, C. (2008b). Rehabilitation with 
Brain-Computer Interface Systems, Computer, Vol. 41, No. 10, 58 

Pfurtscheller, G.; Miiller-Putz, G. R.; Schlogl, A.; Graimann, B.; Scherer, R.; Leeb, R.; 
Brunner, C.; Keinrath, C.; Lee, F.; Townsend, G.; Vidaurre, C. & Neuper, C. (2006c). 
15 Years of BCI Research at Graz University of Technology: Current Projects, IEEE 
Trans Neural Syst Rehabil Eng, Vol. 14, No. 2, 205-10 

Pfurtscheller, G. & Neuper, C. (2010). Dynamics of Sensorimotor Oscillations in a Motor 
Task, In: Brain Computer Interfaces: Revolutionizing Human-Computer Interaction, 
edited by B. Graimann, B. Z. Allison & G. Pfurtscheller. Springer 

Pfurtscheller, G. & Neuper, C. (2001). Motor Imagery and Direct Brain-Computer 
Communication, Proceedings of the IEEE, Vol. 89, No. 7, 1123-34 

Pfurtscheller, G. & Solis-Escalante, T. (2009). Could the Beta Rebound in the EEG Be Suitable 
to Realize a "Brain Switch"? Clin Neurophysiol, Vol. 120, No. 1, 24-9 

Pfurtscheller, G.; Solis-Escalante, T.; Ortner, R.; Linortner, P. & Miiller-Putz, G. R. (2010b). 
Self-Paced Operation of an SSVEP -Based Orthosis with and without an Imagery- 
Based "Brain Switch": A Feasibility Study Towards a Hybrid BCI., IEEE Trans 
Neural Syst Rehabil Eng 


www.intechopen.com 


20 Virtual Reality 


Pineda, J. A.; Silverman, D. S.; Vankov, A. & Hestenes, J. (2003). Learning to Control Brain 
Rhythms: Making a Brain-Computer Interface Possible, IEEE Trans Neural Syst 
Rehabil Eng, Vol. 11, No. 2, 181-4 

Ron Angevin, R.; Diaz-Estrella) A. & Reyes-Lecuona, A. (2005). Development of a 
Braincomputer Interface Based on Virtual Reality to Improve Training Techniques, 
Cyberpsychol Behav, Vol. 8, No. 4, 353-54 

Scherer, R. (2004-2006). RtsBCI: Graz brain-computer interface real-time open source 
package. Available at http:/ /sourceforge.net/ projects/biosig 

Scherer, R.; Lee, F.; Schlégl, A.; Leeb, R.; Bischof, H. & Pfurtscheller, G. (2008). Toward Self- 
Paced Brain-Computer Communication: Navigation through Virtual Worlds, IEEE 
Trans Biomed Eng, Vol. 55, No. 2, 675-82 

Slater, M.; Steed, A. & Chrysanthou, Y. (2002). Computer Graphics and Virtual Environments: 
From Realism to Real-Time. Addison-Wesley, Harlow, UK 

Slater, M. & Usoh, M. (1993). Presence in Immersive Virtual Environments, Proc. IEEE 
Virtual Reality Annual International Symposium, Seattle, WA, 90-96 

Townsend, G.; Graimann, B. & Pfurtscheller, G. (2004). Continuous EEG Classification 
During Motor Imagery-Simulation of an Asynchronous BCI, IEEE Trans Neural Syst 
Rehabil Eng, Vol. 12, No. 2, 258-65 

Usoh, M. & Slater, M. (1995). An Exploration of Immersive Virtual Environments, Endeavour, 
Vol. 19, No. 1, 34-38 

Wolpaw, J. R.; Birbaumer, N.; McFarland, D. J.; Pfurtscheller, G. & Vaughan, T. M. (2002). 
Brain-Computer Interfaces for Communication and Control, Clin Neurophysiol, Vol. 
113, No. 6, 767-91 


www.intechopen.com 


Virtual Reality 


YEDE Edited by Prof. Jae-Jin Kim 


Edred Sy Jaehn Om 


ISBN 978-953-307-518-1 

Hard cover, 684 pages 

Publisher InTech 

Published online 08, January, 2011 
Published in print edition January, 2011 


Technological advancement in graphics and other human motion tracking hardware has promoted pushing 
"virtual reality" closer to "reality" and thus usage of virtual reality has been extended to various fields. The most 
typical fields for the application of virtual reality are medicine and engineering. The reviews in this book 
describe the latest virtual reality-related knowledge in these two fields such as: advanced human-computer 
interaction and virtual reality technologies, evaluation tools for cognition and behavior, medical and surgical 
treatment, neuroscience and neuro-rehabilitation, assistant tools for overcoming mental illnesses, educational 
and industrial uses In addition, the considerations for virtual worlds in human society are discussed. This book 
will serve as a state-of-the-art resource for researchers who are interested in developing a beneficial 
technology for human society. 


How to reference 
In order to correctly reference this scholarly work, feel free to copy and paste the following: 


Gert Pfurtscheller, Robert Leeb, Josef Faller and Christa Neuper (2011). Brain-Computer Interface Systems 
Used for Virtual Reality Control, Virtual Reality, Prof. Jae-Jin Kim (Ed.), ISBN: 978-953-307-518-1, InTech, 
Available from: http:/www.intechopen.com/books/virtual-reality/brain-computer-interface-systems-used-for- 
virtual-reality-control 


INTECH 


open science | open minds 


InTech Europe InTech China 

University Campus STeP Ri Unit 405, Office Block, Hotel Equatorial Shanghai 

Slavka Krautzeka 83/A No.65, Yan An Road (West), Shanghai, 200040, China 

51000 Rijeka, Croatia HEATERS | BER RABA RIE DAR405 SIT 
Phone: +385 (51) 770 447 Phone: +86-21-62489820 

Fax: +385 (51) 686 166 Fax: +86-21-62489821 


www.intechopen.com 


© 2011 The Author(s). Licensee IntechOpen. This chapter is distributed 
under the terms of the Creative Commons Attribution-NonCommercial- 
ShareAlike-3.0 License, which permits use, distribution and reproduction for 
non-commercial purposes, provided the original is properly cited and 
derivative works building on this content are distributed under the same 
license. 


